System and method for determining a contextual insight and generating an interface with recommendations based thereon

ABSTRACT

A system and method for generating an interface for providing recommendations based on contextual insights, the method including: generating at least one signature for at least one multimedia content element identified within an interaction between a plurality of users; generating at least one contextual insight based on the generated at least one signature and user interests of the plurality of users, wherein each contextual insight indicates a current user preference; searching for at least one content item that matches the at least one contextual insight; and generating an interface for providing the at least one content item within the interaction between the plurality of users.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/426,247 filed on Nov. 24, 2016. This application is also acontinuation-in-part of U.S. patent application Ser. No. 15/206,711filed on Jul. 11, 2016, now pending, which claims the benefit of U.S.Provisional Application No. 62/274,295 filed on Jan. 3, 2016. The Ser.No. 15/206,711 application is also a continuation-in-part of U.S. patentapplication Ser. No. 14/280,928 filed on May 19, 2014, now pending,which claims the benefit of U.S. Provisional Application No. 61/833,028filed on Jun. 10, 2013. The Ser. No. 14/280,928 application is also acontinuation-in-part of U.S. patent application Ser. No. 13/856,201filed on Apr. 3, 2013, now pending, which claims the benefit of U.S.Provisional Application No. 61/766,016 filed on Feb. 18, 2013. The Ser.No. 14/280,928 application is also a continuation-in-part of U.S. patentapplication Ser. No. 13/624,397 filed on Sep. 21, 2012, now U.S. Pat.No. 9,191,626. The Ser. No. 13/624,397 application is acontinuation-in-part of:

(a) U.S. patent application Ser. No. 13/344,400 filed on Jan. 5, 2012,now U.S. Pat. No. 8,959,037, which is a continuation of U.S. patentapplication Ser. No. 12/434,221 filed on May 1, 2009, now U.S. Pat. No.8,112,376. The Ser. No. 12/434,221 application is a continuation-in-partof the below-referenced U.S. patent application Ser. Nos. 12/084,150 and12/195,863;

(b) U.S. patent application Ser. No. 12/195,863 filed on Aug. 21, 2008,now U.S. Pat. No. 8,326,775, which claims priority under 35 USC 119 fromIsraeli Application No. 185414, filed on Aug. 21, 2007, and which isalso a continuation-in-part of the below-referenced U.S. patentapplication Ser. No. 12/084,150; and

(c) U.S. patent application Ser. No. 12/084,150 having a filing date ofApr. 7, 2009, now U.S. Pat. No. 8,655,801, which is the National Stageof International Application No. PCT/IL2006/001235 filed on Oct. 26,2006, which claims foreign priority from Israeli Application No. 171577filed on Oct. 26, 2005, and Israeli Application No. 173409 filed on Jan.29, 2006.

All of the applications referenced above are incorporated by referencefor all that they contain.

TECHNICAL FIELD

The present disclosure relates generally to the analysis of multimediacontent, and more specifically to a system for generating an interfaceand providing graphical displays of content to users based on userinteractions.

BACKGROUND

With the abundance of data made available through various means ingeneral and the Internet and world-wide web (WWW) in particular, a needto understand likes and dislikes of users has become essential foron-line businesses.

Existing solutions provide several tools to identify users' preferences.Some of these existing solutions actively require an input from theusers to specify their interests. However, profiles generated for usersbased on their inputs may be inaccurate, as the users tend to provideonly their current interests, or otherwise only provide partialinformation due to privacy concerns.

Other existing solutions passively track the users' activity throughparticular web sites such as social networks. The disadvantage with suchsolutions is that typically limited information regarding the users isrevealed, as users tend to provide only partial information due toprivacy concerns. For example, users creating an account on Facebook®provide in most cases only the minimum information required for thecreation of the account. Additional information about such users may becollected over time, but may take significant amounts of time (i.e.,gathered via multiple social media or blog posts over a time period ofweeks or months) to be useful for accurate identification of userpreferences.

It would therefore be advantageous to provide a solution that overcomesthe deficiencies of the prior art.

SUMMARY

A summary of several example embodiments of the disclosure follows. Thissummary is provided for the convenience of the reader to provide a basicunderstanding of such embodiments and does not wholly define the breadthof the disclosure. This summary is not an extensive overview of allcontemplated embodiments, and is intended to neither identify key orcritical elements of all embodiments nor to delineate the scope of anyor all aspects. Its sole purpose is to present some concepts of one ormore embodiments in a simplified form as a prelude to the more detaileddescription that is presented later. For convenience, the term “someembodiments” may be used herein to refer to a single embodiment ormultiple embodiments of the disclosure.

The embodiments disclosed herein include a method for generating aninterface for providing recommendations based on contextual insights,the method including: generating at least one signature for at least onemultimedia content element identified within an interaction between aplurality of users; generating at least one contextual insight based onthe generated at least one signature and user interests of the pluralityof users, wherein each contextual insight indicates a current userpreference; searching for at least one content item that matches the atleast one contextual insight; and generating an interface for providingthe at least one content item within the interaction between theplurality of users.

The embodiments disclosed herein also include a non-transitorycomputer-readable medium having stored thereon instructions for causinga processing circuitry to perform a method for generating an interfacefor providing recommendations based on contextual insights, the methodincluding: generating at least one signature for at least one multimediacontent element identified within an interaction between a plurality ofusers; generating at least one contextual insight based on the generatedat least one signature and user interests of the plurality of users,wherein each contextual insight indicates a current user preference;searching for at least one content item that matches the at least onecontextual insight; and generating an interface for providing the atleast one content item within the interaction between the plurality ofusers.

The embodiments disclosed herein also include a system for generating aninterface for providing recommendations based on contextual insights.The system comprises: a processing circuitry; and a memory, wherein thememory contains instructions that, when executed by the processingcircuitry, configure the system to: generate at least one signature forat least one multimedia content element identified within an interactionbetween a plurality of users; generate at least one contextual insightbased on the generated at least one signature and user interests of theplurality of users, wherein each contextual insight indicates a currentuser preference; search for at least one content item that matches theat least one contextual insight; and generate an interface for providingthe at least one content item within the interaction between theplurality of users.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter that is regarded disclosed herein is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other objects, features, andadvantages of the disclosed embodiments will be apparent from thefollowing detailed description taken in conjunction with theaccompanying drawings.

FIG. 1 is a network diagram utilized to describe the various disclosedembodiments.

FIG. 2 is a block diagram of an interest analyzer utilized to describethe various disclosed embodiments.

FIG. 3 is a flowchart illustrating a method for profiling userinterests.

FIG. 4 is a flowchart illustrating a method for generating contextualinsights based on analysis of a user's interests and a multimediacontent element according to an embodiment.

FIG. 5 is a block diagram depicting the basic flow of information in thesignature generator system.

FIG. 6 is a diagram showing the flow of patches generation, responsevector generation, and signature generation in a large-scalespeech-to-text system.

FIG. 7 is a flowchart illustrating a method for providingrecommendations for multimedia content elements to a user based oncontextual insights according to an embodiment.

FIG. 8 is a flowchart illustrating a method for generating an interfacebased on contextual insights according to an embodiment.

DETAILED DESCRIPTION

It is important to note that the embodiments disclosed herein are onlyexamples of the many advantageous uses of the innovative teachingsherein. In general, statements made in the specification of the presentapplication do not necessarily limit any of the various claimedembodiments. Moreover, some statements may apply to some inventivefeatures but not to others. In general, unless otherwise indicated,singular elements may be in plural and vice versa with no loss ofgenerality. In the drawings, like numerals refer to like parts throughseveral views.

Certain embodiments disclosed herein include a system and method forgenerating interfaces based on contextual insights. Contextual insightsare generated based on user interests of a user and multimedia contentelements captured or otherwise provided by the user (via, e.g., a userdevice).

The contextual insights are conclusions related to a current preferenceof users. Generating the contextual insights may further includesretrieving a user profile from a database of user profiles and analyzingthe captured multimedia content element. The database is created basedon collection and storage of user interests. The multimedia contentelement is analyzed by generating one or more signatures to themultimedia content element. Based on the signatures, one or moreconcepts of the multimedia content element are determined. Based on thedetermined concepts and the user interests, at least one contextualinsight is generated.

An interface is generated based on the contextual insights.Specifically, the interface includes one or more recommendations ofrecommended content items. The recommendations may include, but are notlimited to, recommendations for multimedia content elements,recommendations for web sites or pages (via, e.g., a hyperlink to a webpage), recommendations for topics of interest (to be, e.g., utilized asa query or to customize a user profile), combinations thereof, and thelike.

As a non-limiting example, if a user captured an image determined as aself-portrait photograph (typically referred to as a “selfie”) andshared the image with another user via user interactions during a chatconversation, and user interests of the user include “fashion,” linksthrough which the users can purchase clothing items that fit the users'preferences an interface, such as an updated chat window, is generatedand provided.

A user interest may be determined, in part, based on the period of timethe users viewed or interacted with the multimedia content elements; atextual input provided by the users, a voice input provided by theusers, a multimedia content element provided by the users, a gesturereceived by the user devices such as, a mouse click, a mouse scroll, atap, and any other gesture on a device having, e.g., a touch screendisplay or a pointing device; content viewed by the user device; and thelike. User interests may further be generated at least partially basedon personal parameters associated with the users, for example,demographic information related to users. The personal parameters may beidentified in, e.g., a user profile associated with each of the users.According to another embodiment, a user interest may be determined basedon a match between a plurality of multimedia content elements viewed bya user and their respective impressions. According to yet anotherembodiment, a user interest may be generated based on multimedia contentelements that the user uploads or shares on the web, such as socialnetworking websites. It should be noted that the user interest may bedetermined based on one or more of the above identified techniques.

FIG. 1 shows a network diagram 100 utilized to describe the variousdisclosed embodiments. As illustrated in FIG. 1 , a network 110 enablesthe communication between different parts of the system. The network 110may be the Internet, the world-wide-web (WWW), a local area network(LAN), a wide area network (WAN), a metro area network (MAN), and othernetworks capable of enabling communication between the elements of thesystem 100.

Further connected to the network 110 is one or more user devices 120-1to 120-n, where ‘n’ is an integer equal to or greater than 1(collectively referred to hereinafter as user devices 120 or user device120, merely for simplicity purposes). The user device 120 may be, forexample, a personal computer (PC), a personal digital assistant (PDA), amobile phone, a tablet computer, a smartphone, a wearable computingdevice, and the like. In some embodiments, the user device 120 may haveinstalled therein an interest analyzer 125. The interest analyzer 125may be a dedicated application, script, or any program code stored in amemory of the user device 120 and is executable, for example, by aprocessing circuitry (e.g., microprocessor) of the user device 120. Theinterest analyzer 125 may be configured to perform some or all of theprocesses performed by a server 130 and disclosed herein.

In another embodiment, the user device 120 may include a local storage127. The local storage 127 may include multimedia content captured orreceived by the user device 120. For example, the local storage 127 mayinclude photographs and videos either captured via a camera (not shown)of the user device 120 or downloaded from a website (e.g., via thenetwork 110).

The user device 120 is configured to at least capture and providemultimedia content elements to the server 130 connected to the network110. The content displayed on a user device 120 may be downloaded fromone of a plurality of web sources 150 (collectively referred tohereinafter as web sources 150 or individually as a web source 150,merely for simplicity purposes), may be embedded in a web-page displayedon the user device 120, or a combination thereof. The uploadedmultimedia content element can be locally saved in the user device 120or can be captured by the user device 120. For example, the multimediacontent element may be an image captured by a camera installed in theuser device 120, a video clip saved in the user device 120, and so on. Amultimedia content element may be, for example, an image, a graphic, avideo stream, a video clip, an audio stream, an audio clip, a videoframe, a photograph, portions thereof, or combinations thereof.

The system 100 also includes the plurality of web sources 150-1 through150-m, where ‘m’ is an integer equal to or greater than 1 (collectivelyreferred to hereinafter as web sources 150 or individually as a websource 150, merely for simplicity purposes) connected to the network110. Each of the web sources 150 may be, for example, a web server, anapplication server, a data repository, a database, a website, ane-commerce website, a content website and the like. The web sources 150include multimedia content elements utilized for generating contextualinsights. Alternatively or collectively, the multimedia content elementsutilized for generating contextual insights may be stored in the localstorage 127 of the user device 120, a storage of the server 130, orboth.

The various embodiments disclosed herein may be realized using theserver 130 and a signature generator system (SGS) 140. The server 130 isconfigured to create a profile for each user of a user device 120 aswill be discussed below.

The SGS 140 is configured to generate a signature based on an analysisof an interaction between a plurality of users, such as multimediacontent within a text chat, video chat, audio call, a combinationthereof, and the like. The process for generating the signatures isexplained in more detail herein below with respect to FIGS. 5 and 6 .Each of the server 130 and the SGS 140 typically includes a processingcircuitry, such as a processor (not shown) that is communicativelyconnected to a memory. The memory typically contains instructions thatcan be executed by the processing circuitry. The server 130 alsoincludes an interface (not shown) to the network 110. In an embodiment,the SGS 140 can be integrated in the server 130. In an embodiment, theserver 130, the SGS 140, or both may include a plurality ofcomputational cores having properties that are at least partlystatistically independent from other cores of the plurality ofcomputational cores. The computational cores are further discussedbelow.

According to an embodiment, a tracking agent or other means forcollecting information through the user device 120 may be configured toprovide the server 130 with tracking information related to themultimedia content element viewed or uploaded by the user and related tothe interaction of the user with another user. The information mayinclude, but is not limited to, the multimedia content element (or a URLreferencing the multimedia content element), the amount of time the useror users have viewed the multimedia content element, a user gesture madewith respect to the multimedia content element, a URL of a webpage inwhich the element was viewed or uploaded to, a combination thereof, andso on. The tracking information is provided for each multimedia contentelement viewed on or uploaded via the user device 120.

The server 130 is configured to determine a user impression with respectto the received tracking information. The user impression may bedetermined for each multimedia content element or for a group ofmultimedia content elements. As noted above, the user impressionindicates the user's attention with respect to a multimedia contentelement or group of multimedia content elements. In one embodiment, theserver 130 may first filter the tracking information to remove detailsthat are not helpful in the determination of the user impression. A userimpression may be determined based on, but not limited to, a click on anelement, a scroll, hovering over an element with a mouse, a change involume, one or more key strokes, and so on. The user impression mayfurther be determined to be either positive (i.e., demonstrating that auser is interested in the impressed element) or negative (i.e.,demonstrating that a user is not particularly interested in theimpressed element). According to one embodiment, a filtering operationmay be performed in order to analyze only meaningful impressions.Impressions may be determined as meaningless and thereby ignored, if,for example, a value associated with the impression is below apredefined threshold.

For example, in an embodiment, if the user hovered over the elementusing his mouse for a very short time (e.g., less than 0.5 seconds),then such a measure is ignored. To this end, in a further embodiment,the server 130 is configured to compute a quantitative measure for theimpression. In one embodiment, for each input measure that is tracked bythe tracking agent, a predefined number is assigned. For example, adwell time over the multimedia content element of 2 seconds or less maybe assigned with a ‘5’; whereas a dwell time of over 2 seconds may beassigned with the number ‘10’. A click on the element may increase thevalue of the quantitative measure by assigning another quantitativemeasure of the impression. After one or more input measures of theimpression have been made, the numbers related to the input measuresprovided in the tracking information are accumulated. The total of theseinput measures is the quantitative measure of the impression.Thereafter, the server compares the quantitative measure to a predefinedthreshold, and if the number exceeds the threshold, the impression isdetermined to positive. In a further embodiment, the input measurevalues may be weighted.

For example, in an embodiment, if a user hovers over the multimediacontent element for less than 2 seconds but then clicks on the element,the score may be increased from 5 to 9 (i.e., the click may add 4 to thetotal number). In that example, if a user hovers over the multimediacontent element for more than 2 seconds and then clicks on the element,the score may be increased from 10 to 14. In some embodiments, theincrease in score may be performed relative to the initial size of thescore such that, e.g., a score of 5 will be increased less (for example,by 2) than a score of 10 would be increased (for example, by 4).

The multimedia content element or elements that are determined as havinga positive user impression are sent to the SGS 140. The SGS 140 is thenconfigured to generate at least one signature for each multimediacontent element or for each portion thereof. The generated signature(s)may be robust to noise and distortions as discussed below.

It should be appreciated that the signatures may be used for profilingthe user's interests, because signatures typically allow for moreaccurate reorganization of multimedia content elements in comparisonthan, for example, utilization of metadata. The signatures generated bythe SGS 140 for the multimedia content elements allow for recognitionand classification of multimedia content elements such ascontent-tracking, video filtering, multimedia taxonomy generation, videofingerprinting, speech-to-text, audio classification, elementrecognition, video/image search and any other application requiringcontent-based signatures generation and matching for large contentvolumes such as, web and other large-scale databases. For example, asignature generated by the SGS 140 for a picture showing a car enablesaccurate recognition of the model of the car from any angle at which thepicture was taken.

In one embodiment, the generated signatures are matched against adatabase of concepts (not shown) to identify a concept structure(hereinafter referred to as a “concept”) that can be associated with thesignature and, thus, with the multimedia content element. For example,an image of a tulip would be associated with a concept of flowers. Aconcept (or a matching concept) is a collection of signaturesrepresenting a multimedia content element and metadata describing theconcept. The collection of signatures is a signature reduced clustergenerated by inter-matching signatures generated for the plurality ofmultimedia content elements. The techniques for generating concepts,concept structures, and a concept-based database are disclosed in U.S.patent application Ser. No. 13/766,463, filed on Feb. 13, 2013, now U.S.Pat. No. 9,031,999, assigned to the common assignee, which is herebyincorporated by reference.

Based on the identified concepts, the server 130 is configured to createor update the user profile. That is, for each user, when a number ofsimilar or identical concepts for multiple multimedia content elementshave been identified over time, the user's preference or interest can beestablished. The interest may be saved to a user profile created for theuser. Whether two concepts are sufficiently similar or identical may bedetermined by, e.g., performing concept matching between the concepts.The matching concept is represented using at least one signature.Techniques for concept matching are disclosed in U.S. patent applicationSer. No. 14/096,901, filed on Dec. 4, 2013, assigned to common assignee,which is hereby incorporated by reference for all the useful informationit contains.

For example, a concept of flowers may be determined as associated with auser interest in ‘flowers’ or ‘gardening.’ In one embodiment, the userinterest may simply be the identified concept. In another embodiment,the interest may be determined using an association table whichassociates one or more identified concepts with a user interest. Forexample, the concept of ‘flowers’ and ‘spring’ may be associated withthe interest of ‘gardening’. Such an association table may be maintainedin the server 130 or in the data warehouse 160.

In an embodiment, the server 130 is configured to generate contextualinsights based on the user's interests and analysis of one or moremultimedia content elements provided by the user device 120. In anexample implementation, the multimedia content elements are providedpursuant to user interactions between a user of the user device 120 andone or more other users. As a non-limiting example, the multimediacontent elements may include, but are not limited to, textual portionsof communications (e.g., emails, chat messages, instant messages, SMSmessages, etc.), audio communications, video communications, multimediacontent shared via communications (e.g., images or videos sent via emailor chat conversation), multimedia content linked to duringcommunications, and the like.

Contextual insights are conclusions related to a current preference ofusers indicated in an interaction. Upon receiving at least onemultimedia content element from the user device 120, at least onesignature is generated for the received multimedia content element. Thesignature is generated by the SGS 140 utilized by the server 130.According to an embodiment, the server 130 is configured to determine aconcept based on the at least one generated signature.

The server 130 queries the user profile stored in the database 160 todetermine at least one user interest based on the determined concept.Based on a response to the query, the server 130 is configured togenerate a contextual insight for the at least one user interest and theat least one signature. The server 130 is configured to search for oneor more content items matching the contextual insight. The search mayinclude querying one or more of the plurality of web sources 150. Thecontent items may include multimedia content elements found during thesearch.

In an embodiment, the server 130 is configured to generate an interfacebased on the content items. The interface may include, but is notlimited to, the content items, a link to the content items, and thelike. Generating the interface may include creating a new interface(e.g., generating a new window to be overlaid on an existing window),updating an existing interface (e.g., adding content items to a chatwindow), or both. The generated interface is sent for display on theuser device 120.

It should be noted that an interaction may change throughoutcorrespondence between the users and therefore, continuous monitoring ofthe interaction and subsequent generation of contextual insights may berequired. As a non-limiting example, an interaction of a user with afamily member may range from joyful to sad, and different types ofcontent may be recommended based on the currently determined contextualinsight.

It should be noted that certain tasks performed by the server 130 andthe SGS 140 may be carried out, alternatively or collectively, by theuser device 120 and the interest analyzer 125. Specifically, in anembodiment, signatures may be generated by a signature generator (notshown in FIG. 1 ) of the user device 120. In another embodiment, theinterest analyzer 125 may be configured to generate contextual insightsand to search for content items matching the contextual insights. Theinterest analyzer 125 may be further configured to identify matchingcontent items and to cause a display of the matching content items onthe user device 120 as recommendations. An example block diagram of aninterest analyzer 125 installed on a user device 120 is describedfurther herein below with respect to FIG. 2 .

It should further be noted that the signatures may be generated formultimedia content elements stored in the web sources 150, in the localstorage 127 of the user device 120, or a combination thereof.

FIG. 2 depicts an example block diagram of an interest analyzer 125installed on the user device 120 according to an embodiment. Theinterest analyzer 125 may be configured to access an interface a userdevice or a server. The interest analyzer 125 is further communicativelyconnected to a processing circuitry (e.g., a processing circuitry of theuser device 120, not shown) such as a processor and to a memory (e.g., amemory of the user device 120, not shown). The memory contains thereininstructions that, when executed by the processing circuitry, configuresthe interest analyzer 125 as further described hereinabove and below.The interest analyzer 125 may further be communicatively connected to astorage unit (e.g., the storage 127 of the user device 120, not shown)including a plurality of multimedia content elements.

In an embodiment, the interest analyzer 125 includes a signaturegenerator (SG) 210, a data storage (DS) 220, and a recommendationsengine 230. The signature generator 210 may be configured to generatesignatures for multimedia content elements. In a further embodiment, thesignature generator 210 includes a plurality of computational cores asdiscussed further herein above, where each computational core is atleast partially statistically independent of the other computationscores.

The data storage 220 may store a plurality of multimedia contentelements, a plurality of concepts, signatures for the multimedia contentelements, signatures for the concepts, or a combination thereof. In afurther embodiment, the data storage 220 may include a limited set ofconcepts relative to a larger set of known concepts. Such a limited setof concepts may be utilized when, for example, the data storage 220 isincluded in a device having a relatively low storage capacity such as,e.g., a smartphone or other mobile device, or otherwise when lowermemory use is desirable.

The recommendations engine 230 may be configured to generate contextualinsights based on multimedia content elements related to the userinterest, to query sources of information (including, e.g., the datastorage 220 or another data source), and to cause a display ofrecommendations on the user device 120.

According to an embodiment, the interest analyzer 125 is configured toreceive at least one multimedia content element. The interest analyzer125 is configured to initialize a signatures generator (SG) 210 togenerate at least one signature for the received at least one multimediacontent element. The memory further contains instructions to query auser profile of the user stored in a data storage (DS) 220 to determinea user interest. The memory further contains instructions to generate acontextual insight based on the user interest and the at least onesignature. Based on the contextual insight, a recommendations engine 230is initialized to search for one or more content items that match thecontextual insight. The matching content items may be provided by therecommendations engine 230 to the user as recommendations via theinterface.

Each of the recommendations engine 230 and the signature generator 210can be implemented with any combination of general-purposemicroprocessors, multi-core processors, microcontrollers, digital signalprocessors (DSPs), field programmable gate array (FPGAs), programmablelogic devices (PLDs), controllers, state machines, gated logic, discretehardware components, dedicated hardware finite state machines, or anyother suitable entities that can perform calculations or othermanipulations of information.

In certain implementations, the recommendation engine 230, the signaturegenerator 210, or both can be implemented using an array ofcomputational cores having properties that are at least partlystatistically independent from other cores of the plurality ofcomputational cores. The computational cores are further discussedbelow.

According to another implementation, the processes performed by therecommendation engine 230, the signature generator 210, or both can beexecuted by a processing circuitry of the user device 120 or of theserver 130. Such processing circuitry may include machine-readable mediafor storing software. Software shall be construed broadly to mean anytype of instructions, whether referred to as software, firmware,middleware, microcode, hardware description language, or otherwise.Instructions may include code (e.g., in source code format, binary codeformat, executable code format, or any other suitable format of code).The instructions, when executed by the one or more processors, cause theprocessing circuitry to perform the various functions described herein.

It should be noted that, although FIG. 2 is described with respect to aninterest analyzer 125 included in the user device 120, any or all of thecomponents of the interest analyzer 125 may be included in anothersystem or systems (e.g., the server 130, the signature generator system140, or both) and utilized to perform some or all of the tasks describedherein without departing from the scope of the disclosure. As anexample, the interest analyzer 125 operable in the user device 120 maysend multimedia content elements to the signature generator system 140and may receive corresponding signatures therefrom. As another example,the user device 120 may send signatures to the server 130 and mayreceive corresponding recommendations or concepts therefrom. As yetanother example, the interest analyzer 125 may be included in the server130 and may provide recommendations to the user device 120 based onmultimedia content elements identified by or received from the userdevice 120.

FIG. 3 depicts an example flowchart 300 illustrating a method forcreating user profiles according to an embodiment. It should be notedthat, in an embodiment, tracking information is collected by a userdevice. In various embodiments, tracking information may be collectedfrom other sources such as, e.g., a database. In an embodiment, themethod may be performed by a server (e.g., the server 130).

At S310, tracking information is obtained via a user device (e.g., theuser device 120-1). In an embodiment, the obtained tracking informationmay be received from, e.g., an agent installed on the user device andconfigured to collect tracking information. In a further embodiment,S310 may include filtering the tracking information. As noted above, thetracking information is collected with respect to a multimedia contentelement displayed over the user device. In an embodiment, the trackinginformation may include, but is not limited to, the multimedia contentelement (or a link thereto) displayed on the user device and usergestures with respect to displayed multimedia content element. In anembodiment, the tracking information may be collected via a web browserexecuted by the user device.

At S320, one or more user impressions is determined based on theobtained tracking information. In a further embodiment, each userimpression may be assigned a score based on a value of the user gesturesutilized to determine the user impression. The score may further bepositive or negative. In yet a further embodiment, S320 may includefiltering the user impressions so as to only determine meaningfulimpressions. The filtering may include, for example, filtering out anyuser impressions associated with a score that is below a predefinedthreshold.

The user impressions may be determined based on user gestures such as,but not limited to, a click on an element, a scroll, hovering over anelement with a mouse, a change in volume, one or more key strokes, acombination thereof, and so on. The user impressions may further bedetermined to be either positive (i.e., demonstrating that a user isinterested in the impressed element) or negative (i.e., demonstratingthat a user is not particularly interested in the impressed element).One embodiment for determining the user impression is described hereinabove. The user impression is determined for one or more multimediacontent elements identified in the tracking information.

At S330, it is checked if any of the user impressions is positive and,if so, execution continues with S340; otherwise, execution continueswith S380. Whether a user impression is positive is discussed furtherherein above with respect to FIG. 1 .

At S340, at least one signature is generated for each multimedia contentelement that is associated with a positive user impression. As notedabove, the tracking information may include the actual multimediacontent element or a link thereto. In the latter case, the multimediacontent element is first retrieved from its location. The at least onesignature for the multimedia content element may be generated by a SGS(e.g., the SGS 140) as described further herein below.

At S350, one or more concepts related to the multimedia content elementsassociated with positive user impressions is determined. In anembodiment, S350 includes querying a concept-based database using thegenerated signatures. In a further embodiment, S350 may include matchingthe generated signatures to at least one signature associated withconcepts in the concept-based database. In yet a further embodiment,each of the concepts may be associated with one or more particularportions of the multimedia content element. As an example, a multimediacontent element image of a man wearing a baseball shirt may beassociated with the concept “baseball fan,” and the portions of theimage related to the man may be associated with the concept “man” andthe portions of the image related to the shirt may be associated withthe concept “sports clothing” or “baseball.”

At S360, based on the determined concepts, the user interest isdetermined. Determining the user interest may include, but is notlimited to, identifying a positive user impression with respect to anyof the concepts. In an embodiment, the user interest may be furtherdetermined with respect to particular portions of the multimedia contentelement and user gestures related to those particular portions. Forexample, if a multimedia content element is an image showing a dog and acat, a click on a portion of the image showing the dog may indicate apositive impression (and, therefore, a user interest), in “dogs” but notnecessarily a user interest in “cats.”

As a non-limiting example of determining user interest, the user views aweb-page that contains an image of a car. The image is then analyzed anda signature is generated respective thereto. As it appears that the userspent time above a certain threshold viewing the image of the car, theuser's impression is determined as positive. It is therefore determinedthat a user interest is “cars.”

At S370, the determined user interest is saved as part of a user profilefor the user in a database (e.g., the database 160). It should be notedthat if no user profile for the user exists in the database, a userprofile may be created for the user. It should be noted that a uniqueuser profile may be created for each user of a user device. The user maybe identified by a unique identification number assigned, for example,by the tracking agent. The unique identification number typically doesnot reveal the user's identity. The user profile can be updated overtime as additional tracking information is gathered and analyzed by theserver. In an embodiment, the server 130 analyzes the trackinginformation only when a sufficient amount of additional trackinginformation has been collected.

At S380, it is determined whether additional tracking information isreceived and, if so, execution continues with S310; otherwise, executionterminates. As noted above, in an embodiment, S380 may includedetermining whether a sufficient amount of additional trackinginformation has been received

As a non-limiting example, tracking information including a videofeaturing a cat playing with a toy and a cursor hovering over the catfor 20 seconds is obtained from an agent installed on a user device.Based on the tracking information and, specifically, the cursor hoveringover the cat for more than 5 seconds, it is determined that a userimpression of the video is positive. A signature is generated for thevideo, and a concept of “cats” is determined. Based on the positive userimpression of the concept of “cats,” a user interest in “cats” isdetermined. The user interest is saved as part of a user profile of theuser.

FIG. 4 depicts an example flowchart 400 illustrating a method forgenerating contextual insights according to another embodiment.

At S410, at least one multimedia content element is received. Themultimedia content element may be, for example, an image, a graphic, avideo stream, a video clip, an audio stream, an audio clip, a videoframe, a photograph, combinations thereof, or portions thereof. The atleast one multimedia content element may be captured by a sensorincluded in a user device (e.g., the user device 120).

At S420, at least one signature is generated for each receivedmultimedia content element. The signatures for the multimedia contentelements are typically generated by a SGS (e.g., the SGS 140) asdescribed hereinabove.

At optional S430, at least one concept is determined for each generatedsignature. In an embodiment, S430 includes querying a concept-baseddatabase using the generated signatures. In a further embodiment, thegenerated signatures are matched to signatures representing conceptsstored in the concept-based database, and concepts associated withmatching the generated signatures above a predetermined threshold may bedetermined.

At S440, the generated signatures or the determined concepts are matchedto user interests associated with the user. The user interests may beextracted from a user profile stored in a database (e.g., the database160). In an embodiment, matching the concepts to the user interests mayinclude matching signatures representing the determined concepts tosignatures representing the user interests.

At S450, at least one contextual insight is generated based on a matchbetween the user interest and the concept(s) or signature(s). Thecontextual insights are conclusions related to a preference of the user.For example, if a user interest is “motorcycles” and a concept relatedto multimedia content elements viewed by the user is “red vehicles,” acontextual insight may be a user preference for “red motorcycles.” Asanother example, if a user interest is “shopping” and a concept relatedto multimedia content elements viewed by the user is “located in LasVegas, Nev.,” a contextual insight may be a preference for shoppingoutlets in Las Vegas, Nev.

At S460, it is checked whether additional multimedia content elementsare received and, if so, execution continues with S410; otherwise,execution terminates.

FIGS. 5 and 6 illustrate the generation of signatures for the multimediacontent elements by the SGS 140 according to one embodiment. Anexemplary high-level description of the process for large scale matchingis depicted in FIG. 4 . In this example, the matching is for a videocontent.

Video content segments 2 from a Master database (DB) 6 and a Target DB 1are processed in parallel by a large number of independent computationalCores 3 that constitute an architecture for generating the Signatures(hereinafter the “Architecture”). Further details on the computationalCores generation are provided below. The independent Cores 3 generate adatabase of Robust Signatures and Signatures 4 for Targetcontent-segments 5 and a database of Robust Signatures and Signatures 7for Master content-segments 8. An exemplary and non-limiting process ofsignature generation for an audio component is shown in detail in FIG. 5. Finally, Target Robust Signatures and/or Signatures are effectivelymatched, by a matching algorithm 9, to Master Robust Signatures and/orSignatures database to find all matches between the two databases.

To demonstrate an example of signature generation process, it isassumed, merely for the sake of simplicity and without limitation on thegenerality of the disclosed embodiments, that the signatures are basedon a single frame, leading to certain simplification of thecomputational cores generation. The Matching System is extensible forsignatures generation capturing the dynamics in-between the frames.

The Signatures' generation process is now described with reference toFIG. 5 . The first step in the process of signatures generation from agiven speech-segment is to break down the speech-segment to K patches 14of random length P and random position within the speech segment 12. Thebreakdown is performed by the patch generator component 21. The value ofthe number of patches K, random length P and random position parametersis determined based on optimization, considering the tradeoff betweenaccuracy rate and the number of fast matches required in the flowprocess of the server 130 and SGS 140. Thereafter, all the K patches areinjected in parallel into all computational Cores 3 to generate Kresponse vectors 22, which are fed into a signature generator system 23to produce a database of Robust Signatures and Signatures 4.

In order to generate Robust Signatures, i.e., Signatures that are robustto additive noise L (where L is an integer equal to or greater than 1)by the Computational Cores 3, a frame ‘i’ is injected into all the Cores3. Then, Cores 3 generate two binary response vectors: {right arrow over(S)} which is a Signature vector, and {right arrow over (RS)} which is aRobust Signature vector.

For generation of signatures robust to additive noise, such asWhite-Gaussian-Noise, scratch, etc., but not robust to distortions, suchas crop, shift and rotation, etc., a core Ci={n_(i)} (1≤i≤L) may consistof a single leaky integrate-to-threshold unit (LTU) node or more nodes.The node n_(i) equations are:

$V_{i} = {\sum\limits_{j}{w_{ij}k_{j}}}$ n_(i) = θ(Vi − Thx)

where, θ is a Heaviside step function; w_(ij) is a coupling node unit(CNU) between node i and image component j (for example, grayscale valueof a certain pixel j); k_(j) is an image component ‘j’ (for example,grayscale value of a certain pixel j); Th_(x) is a constant Thresholdvalue, where x is ‘S’ for Signature and ‘RS’ for Robust Signature; andV_(i) is a Coupling Node Value.

The Threshold values Th_(X) are set differently for Signature generationand for Robust Signature generation. For example, for a certaindistribution of values (for the set of nodes), the thresholds forSignature (Th_(S)) and Robust Signature (Th_(RS)) are set apart, afteroptimization, according to at least one or more of the followingcriteria:For: V _(i) >Th _(RS)1−p(V>Th _(S))−1−(1−ε)^(I)<<1  1:

i.e., given that I nodes (cores) constitute a Robust Signature of acertain image I, the probability that not all of these I nodes willbelong to the Signature of a same, but noisy image, Ĩ is sufficientlylow (according to a system's specified accuracy).p(V _(i) >Th _(RS))≈l/L  2:

i.e., approximately I out of the total L nodes can be found to generatea Robust Signature according to the above definition.

3: Both Robust Signature and Signature are generated for certain framei.

It should be understood that the generation of a signature isunidirectional, and typically yields lossless compression, where thecharacteristics of the compressed data are maintained but theuncompressed data cannot be reconstructed. Therefore, a signature can beused for the purpose of comparison to another signature without the needof comparison to the original data. The detailed description of theSignature generation can be found U.S. Pat. Nos. 8,326,775 and8,312,031, assigned to the common assignee, which are herebyincorporated by reference for all the useful information they contain.

A Computational Core generation is a process of definition, selection,and tuning of the parameters of the cores for a certain realization in aspecific system and application. The process is based on several designconsiderations, such as:

-   -   (a) The Cores should be designed so as to obtain maximal        independence, i.e., the projection from a signal space should        generate a maximal pair-wise distance between any two cores'        projections into a high-dimensional space.    -   (b) The Cores should be optimally designed for the type of        signals, i.e., the Cores should be maximally sensitive to the        spatio-temporal structure of the injected signal, for example,        and in particular, sensitive to local correlations in time and        space. Thus, in some cases a core represents a dynamic system,        such as in state space, phase space, edge of chaos, etc., which        is uniquely used herein to exploit their maximal computational        power.    -   (c) The Cores should be optimally designed with regard to        invariance to a set of signal distortions, of interest in        relevant applications.

Detailed description of the Computational Core generation and theprocess for configuring such cores is discussed in more detail in theU.S. Pat. No. 8,655,801 referenced above.

FIG. 7 depicts an example flowchart 700 illustrating a method forproviding recommendations to users based on contextual insightsaccording to an embodiment. It should be noted that, in variousembodiments, recommendations may be provided without first receivingmultimedia content elements to analyze. In such embodiments,recommendations may be determined and provided in response to, e.g., apredetermined event, input from a user, and so on. As a non-limitingexample, a user may request a recommendation for a movie or TV show towatch on a video streaming content website based on his or herinterests.

At S710, at least one contextual insight indicating a preference of theuser is identified. In an embodiment, the at least one contextualinsight may be identified based on, but not limited to, a request for arecommendation, a user profile of the user, a multimedia content elementprovided by the user (via, e.g., a user device), a combination thereof,and the like. In another embodiment, the at least one contextual insightmay be generated as described further herein above with respect to FIG.4 .

At S720, a search for content items matching the identified contextualinsights is performed. The matching content items may include, but arenot limited to, multimedia content elements, web-pages featuringmatching content, electronic documents featuring matching content,combinations thereof, and the like. In an embodiment, S720 may includematching signatures representing the identified contextual insights tosignatures of content items of one or more web sources. As an example,if a contextual insight is a preference for “police dramas,” contentitems related to television and movie dramas prominently featuringpolice and detectives may be found during the search.

At S730, upon identification of at least one matching content item, theat least one matching content item is provided as a recommendation tothe user device. Providing the matching content items as recommendationsmay include, but is not limited to, providing one or more links to eachcontent item, providing identifying information about each content item,sending the content items to the user device, notifying the user ofcontent items existing on the user device, combinations thereof, and soon.

At S740, it is checked whether additional contextual insights areidentified and, if so, execution continues with S720; otherwise,execution terminates.

As a non-limiting example, in case the user is determined as currentlyviewing an image of a vehicle such as a Ford® Focus, and a user profileindicates that he is based in Manhattan, N.Y., a link to a financinginstitution that offers financing plans for purchasing vehicles may befound and provided as a recommendation to the user device.

FIG. 8 is a flowchart illustrating a method 800 for generating a userinterface with recommendations based on contextual insights according toan embodiment.

At S805, an interaction between users is identified. The interaction maybe, for example, a chat interaction between two users. Other types ofinteractions may include, but are not limited to, emails, instantmessages, SMS messages, audio calls, video calls, combinations thereof,and the like.

At S810, multimedia content elements (MMCEs) associated with theinteraction are identified. The multimedia content elements may includecontent within the interaction such as, for example, textual, audio,image, or video content shared within the interaction, a link thereto, aportion thereon, a combination thereof, and the like. When the contentwithin the interaction includes a link to multimedia content elements,S810 may further include retrieving the multimedia content elements fromthe location indicated by the link.

At S815, at least one signature is generated based on each multimediacontent element. The at least one signature for the multimedia contentelement may be generated by a SGS (e.g., the SGS 140) as describedfurther herein above.

At S820, one or more concepts related to the multimedia content elementassociated with the interaction are determined. In an embodiment, S820includes querying a concept-based database using the generatedsignatures. In a further embodiment, S820 may include matching thegenerated signatures to signatures associated with concepts in theconcept-based database. In yet a further embodiment, each of theconcepts may be associated with one or more particular portions of themultimedia content element. As an example, a multimedia content elementimage of a man wearing a baseball shirt may be associated with theconcept “baseball fan,” and the portions of the image related to the manmay be associated with the concept “man” and the portions of the imagerelated to the shirt may be associated with the concept “sportsclothing” or “baseball.”

At S830, one or more contextual insights is generated based on thegenerated signatures and one or more user interests. The contextualinsights may be based on characteristics of the interaction, i.e., thetype of interaction, relationship of the users within the interaction,topics associated with the interaction, interests associated with theinteraction, and the like. The contextual insights may be determined asdiscussed in FIG. 4 . The contextual insights are additionally based onuser interests (e.g., user interests determined as described hereinabove with respect to FIG. 3 ), and may be determined based on userinterests indicated in one or more user profiles. The user profiles maybe accessed via a database.

At S835, based on the determined contextual insights, a search isperformed for one or more recommended content items. The search may beperformed by querying one or more web sources, databases, social mediaplatforms, and the like. The recommended content items may includemultimedia content elements matching the determined contextual insights.

At S840, an interface for providing the recommended content items isgenerated. The generated interface may include, but is not limited to,the recommended content items, pointers to locations of the recommendedcontent items, both, and the like. The generation of the interface maybe based on a type of user device used for the interaction. As anon-limiting example, for an Android® device, a virtual keyboard may begenerated (e.g., a virtual keyboard including keys for selectivelysharing one or more of the recommended content items); for an iPhone®device, a widget may be generated. The generation of the interface mayinclude updating an existing interface, e.g., inserting content into anexisting chat window, or overlaying a new interface, e.g., overlayingthe content over an existing chat window.

At S850, the interface is provided to two or more user devices fordisplay. When the interface is generated based on multimedia contentelements in communications between two or more user devices, theinterface may be provided to each of those user devices.

At S860, it is checked whether additional multimedia content elementshave been identified and if so, execution continues with S810;otherwise, execution terminates. In an embodiment, during userinteractions (e.g., during a chat conversation between users), newcontextual insights may be repeatedly generated based on newinteractions (e.g., based on new text message or shared multimediacontent elements), and new recommended content items may be providedaccordingly.

The various embodiments disclosed herein can be implemented as hardware,firmware, software, or any combination thereof. Moreover, the softwareis preferably implemented as an application program tangibly embodied ona program storage unit or computer readable medium consisting of parts,or of certain devices and/or a combination of devices. The applicationprogram may be uploaded to, and executed by, a machine comprising anysuitable architecture. Preferably, the machine is implemented on acomputer platform having hardware such as one or more central processingunits (“CPUs”), a memory, and input/output interfaces. The computerplatform may also include an operating system and microinstruction code.The various processes and functions described herein may be either partof the microinstruction code or part of the application program, or anycombination thereof, which may be executed by a CPU, whether or not sucha computer or processor is explicitly shown. In addition, various otherperipheral units may be connected to the computer platform such as anadditional data storage unit and a printing unit. Furthermore, anon-transitory computer readable medium is any computer readable mediumexcept for a transitory propagating signal.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the principlesof the disclosure and the concepts contributed by the inventor tofurthering the art, and are to be construed as being without limitationto such specifically recited examples and conditions. Moreover, allstatements herein reciting principles, aspects, and embodiments, as wellas specific examples thereof, are intended to encompass both structuraland functional equivalents thereof. Additionally, it is intended thatsuch equivalents include both currently known equivalents as well asequivalents developed in the future, i.e., any elements developed thatperform the same function, regardless of structure.

What is claimed is:
 1. A method for generating an interface forproviding recommendations based on contextual insights, comprising:obtaining from a user device, tracking information related to aninteraction between a plurality of users; determining, by a server andbased on the tracking information, impressions of the plurality ofusers; classifying, by the server, the impressions to positiveimpressions and negative impressions; determining, by the server, userinterests of the plurality of users based on at least some of thepositive impressions while ignoring the negative impressions;generating, by a plurality of independent computational cores havingproperties that are at least partly statistically independent from othercomputational cores of the plurality of computational cores and belongto a signature generator system that is in communication with theserver, at least one signature for at least one multimedia contentelement identified within the interaction between the plurality ofusers; generating, by the server, at least one contextual insight basedon the generated at least one signature and the user interests of theplurality of users, wherein each contextual insight indicates a currentuser preference; searching, by the server, for at least one content itemthat matches the at least one contextual insight; generating, by theserver, an interface for providing, while displayed on the user device,the at least one content item within the interaction between theplurality of users; and sending the interface to the user device.
 2. Themethod of claim 1, further comprising: determining, based on a userprofile of each of the plurality of users, the user interests of theplurality of users.
 3. The method of claim 1, wherein the at least onecontextual insight is generated based further on a relationship amongthe plurality of users.
 4. The method of claim 1, further comprising:determining, based on the generated at least one signature, at least oneconcept associated with the at least one multimedia content element,wherein the at least one contextual insight is generated further basedon the determined at least one concept.
 5. The method of claim 1,wherein the interaction is any of: a text chat, a video call, and anaudio call.
 6. The method of claim 1, comprising determining that theimpression of the user is positive when the quantitative measure of theimpression exceeds a threshold.
 7. The method according to claim 1wherein the at least one contextual insight comprises a first contextualinsight that indicates preference of a first user of the plurality ofusers and a second contextual insight that indicates preference of asecond user of the plurality of users, wherein the second user differsfrom the first user.
 8. A non-transitory computer readable medium havingstored thereon instructions for causing a processing circuitry of aserver to perform a process, the process comprising: obtaining from auser device, tracking information related to an interaction between aplurality of users; determining, based on the tracking information,impressions of the plurality of users; classifying the impressions topositive impressions and negative impressions; determining userinterests of the plurality of users based on at least some of thepositive impressions while ignoring the negative impressions; generatingby a plurality of independent computational cores having properties thatare at least partly statistically independent from other computationalcores of the plurality of computational cores and belong to a asignature generator system that is in communication with the server, atleast one signature for at least one multimedia content elementidentified within the interaction between the plurality of users;generating at least one contextual insight based on the generated atleast one signature and user interests of the plurality of users,wherein each contextual insight indicates a current user preference;searching for at least one content item that matches the at least onecontextual insight; a generating an interface for providing, whiledisplayed on the user device, the at least one content item within theinteraction between the plurality of users and sending the interface tothe user device.
 9. A system for providing recommendations based on auser interest, comprising: a server that comprises a processingcircuitry and a memory; and a signature generator that is incommunication with the server; wherein the memory contains instructionsthat, when executed by the processing circuitry, configure the systemto: obtain tracking information related to an interaction between aplurality of users; determine, based on the tracking information,impressions of the plurality of users; wherein a determination of animpression a user comprises calculating input measures related to theimpression, and aggregating the input measures to provide a quantitativemeasure of the impression; generating, by a plurality of independentcomputational cores having properties that are at least partlystatistically independent from other computational cores of theplurality of computational cores and belong to the signature generator,at least one signature for at least one multimedia content elementidentified within the interaction between the plurality of users;generate at least one contextual insight based on the generated at leastone signature and user interests of the plurality of users, wherein eachcontextual insight indicates a current user preference; search for atleast one content item that matches the at least one contextual insight;generate an interface for providing, while displayed on the user device,the at least one content item within the interaction between theplurality of users; and sending the interface to the user device. 10.The system of claim 9, the processing circuitry is configured todetermine, based on a user profile of each of the plurality of users,the user interests of the plurality of users.
 11. The system of claim 9,wherein the processing circuitry is configured to generate the at leastone contextual insight based on at least one characteristic of theinteraction, wherein the at least one characteristic of the interactionis a relationship among the plurality of users.
 12. The system of claim9, wherein the processing circuitry is configured to: determine, basedon the generated at least one signature, at least one concept associatedwith the at least one multimedia content element, wherein the at leastone contextual insight is generated further based on the determined atleast one concept.